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FIFTEEN DAILY DRILLS EMPHASIZING MASTERY OF BASIC NUMBER 
FACTS, SUCH AS ARITHMETIC OPERATIONS AND UNITS OF MEASUREMENT 
WERE CONSTRUCTED TO ENCOMPASS 7 PREVIOUSLY REPORTED 
ATTRIBUTES OF AN EFFECTIVE DRILL. ATTRIBUTES ARE MIXED DRILL, 
TIME LIMIT, INCREASINGLY DIFFICULT EXAMFLES, THOROUGH 
COVERAGE, FREQUENT AND SMALL AMOUNTS, VERBAL PROBLEMS, 
DIAGNOSIS FACILITATION, SHORT DRILLS AVERAGING 20 FROBLEMS 
REACHED THE 41 GRADE 4 PUPILS (AVERAGE IQ WAS 130) VIA A 
COMMERCIALLY AVAILABLE TELETYPE CONNECTED FROM THEIR 
CLASSROOM TO A STANFORD UNIVERSITY COMPUTER. LITTLE TEACHER 
PREPARATION WAS REQUIRED. WITH IMMEDIATE FEEDBACK AND A 
10— SECOND TIME LIMIT FER RESPONSE, EACH STUDENT HAD 2 CHANCES 
TO ANSWER CORRECTLY. RANKING THE LESSONS BY PROPORTION OF 
CORRECT RESPONSES AND CLASSIFYING THEM BY TYPE OF FROBLEM 
SHOWED THAT LESSONS OF THE SAME TYPE TEND TO BE GROUPED IN 
ADJACENT RANKS. AMOUNT OF PREVIOUS PRACTICE, AND NUMBER OF 
PUPILS TAKING EACH LESSON CONFOUND THIS RELATIONSHIP, 

HOWEVER. THE RANK-ORDER CORRELATION BETWEEN THE AVERAGE 
PROPORTION OF ERRORS AND THE TIME TAKEN TO COMPLETE A LESSON 
WAS STATISTICALLY SIGNIFICANT. A COMPARISON OF 5 INDIVIDUAL 
STUDENTS ALSO SHOWS THIS RELATIONSHIP. DATA REPORTED HERE 
WERE INTENDED TO GIVE ONLY A SENSE OF METHODS FOR MORE 
RIGOROUS CLASSROOM EXPERIMENTATION. FUTURE EXPANSION OF THE 
PROJECT CALLS FOR 80 STUDENTS USING EACH OF 3 TELETYPES EVERY 
DAY, MORE EXTENSIVE REPORTS TO STUDENTS AND TEACHERS, FROGRAM 
BRANCHING CAPABILITIES. <LH> 
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Computer-based Teletype* 

Patrick Suppes, Max Jerraan, and Guy Groen 

Given a computer-based instructional program, and a school in which 
to work; it is possible to supplement or enrich the teacher's instruction 
by either taking over the more routine daily tasks, presenting special 
materials, or the daily lesson itself. The task of the present project 
was to prepare a program in arithmetic to review and teach the basic 
number facts supplementing the teacher's daily instruction at the 
fourth-grade level. In addition, the project was concerned with gaining 
a clearer picture of what the optimum teacher -machine interaction 
pattern should be in order to take full advantage of both. 



Objectives 

One of the primary objectives was to review and teach the basic 
number facts which comprise an important part of a fourth-grade mathe 
matics program. Often those engaged in teaching "modern mathematics" 
play down this part of the curriculum. Stressing fundamental concepts 
and structure is essential, but mastery of the basic facts should not 
be neglected. 



*This research has been supported by the National Science Foundation 
and the Carnegie Corporation of New York. We are much indebted for the 
cooperation of Mr. Victor Norton, the principal of the school, and Mrs. 
Fran Emery, teacher of the fourth -grade class in Grant School, Cupertino 
School District, in which this study was conducted. 
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In working out a continuous daily program, sequencing of material 
and providing a proper amount of review, so as to correlate with the 
day-to-day classroom instruction, become practical objectives. In our 
case both level of difficulty and length of exercise directly affected 
-the running time for each student, which had to be sufficiently shoru to 
permit each student in the class an opportunity to perform on the machine 
during the regular school day. 

To our knowledge no elementary-school teacher has had a computer- 
•based teaching device available to her on a daily basis up to this time. 
The mechanics of having each child take his turn during the day without 
disrupting the regular work of the class was of some concern. Related 
to this problem was concern over the ability of a teacher untrained in 
the use of the machine or its operation to adjust to its presence in the 
room and to use it optimally. From previous experience in observing 
young children operate the teaching machines in Stanford's Computer- 
Based Instruction Laboratory it was believed that there would be little 
or no problem in the children's adjusting to the machine. (This assump- 
tion was correct; most students were very quick to master the simple 

operations required. ) 



Related Research 

Programmed Instruction . Reports of extensive research over long 
periods of time are lacking. While some creditable work has been done 
in the areas of branching and feedback variables, few studies using 
computer-based teaching devices are available. Most reported studies 
may be summarized by saying that students using well -written programmed 
materials, whether in a text or machine-like device, will be able to 
master some material as well as students in a regular class in somewhat 



less time. 



Drill in Arithmetic . Some confusion still exists in the minds of 
many people concerning the use of drill in today* s curriculum. Brownell 
(1929) as well as many others, has pointed out the need for well-planned 
drill in commenting on readiness for division by stating "If children 
find the topics hard, many times it is due to inadequate mastery of the 
skills and basic facts needed." Some years ago Busswell (1927) reported 
finding that 93 percent of the errors in long division made by fourth-, 
fifth- and sixth-grade pupils was due to a lack of mastery of number 
facts rather than number processes. Spencer (1929) maintained that there 
are indeed "typical errors in arithmetic accounting for 75 percent of 
all the errors in addition, 80 percent of the errors in subtraction, 

50 percent of the errors in multiplication, and 70 percent of the errors 
in division." 

The dangers of teaching by drill methods alone were pointed out in 
a later study by Brownell and Chazel (1960). It was emphasized that 
effective teaching must precede drill, if the drill is to have the desired 
results. It seems apparent on the basis of the foregoing that drills 
can most effectively be used to overcome the large percentage of typical 
errors in arithmetic after an introduction to the subject has been given. 
Also, a student should be given an opportunity to correct errors he 
mak es in his daily work. Suppes and Ginsberg (1962) found that young 
children required to make an overt correction response after an incorrect 
response performed significantly better than a noncorrection group on a 
concept-formation task. 

After studying a large number of r. ports on drill methods, Wilson 
(1925) concluded that to be effective a drill should have the following 
attributes. 

1. It should be on the entire process. 

2. It should come frequently in small amounts. 

3. Each unit should be a mixed drill. 

4. It should have a time limit. 

5. Examples in a drill should be in order of difficulty. 

6. Drills should include verbal problems. 

7» Drills should facilitate diagnosis. 



Procedure 



The Machine and "the Classroom. The "teaching machine used in "this 
project was a commercially available teletype* connected Dy private phone 
line to the computer in Stanford’s Laboratory. A large book closet, 
which opened into the classroom, was modified by adding a ventilation 
fan, light, and electrical outlet. This provided privacy for the user 
an d insulated the rest of the class from the operational noise of the 
teletype. With these very minor modifications, the closet provided an 
excellent teaching station throughout the day. 

The Instruct ional Program . Instruction on the machine began in the 
spring of 1965, running for seven consecutive weeks. The daily drills 
were constructed on the principles of the above— mentioned research, 
particularly in terms of the attributes listed by Wilson# Each drill 
was short, 5 to 6 minutes, varying from 5 to 30 problems (with an average 
of 20). 

As each student took his turn, the machine printed out ’’please type 
your name. 11 The student spelled his name by typing, using the hunt-and- 
peck method for the most part. If his name was incorrectly spelled, he 
was informed "This name is not on the student list, try again. Please 
type your nam e - " A proper entry set the program in operation and the 
first problem was printed out, leaving a blank for the correct response. 
The ma chine was programmed to position itself at the blank so as to have 
the response properly placed. A correct response was reinforced by the 
appearance of the next problem. An incorrect response was indicated by 
the word "wrong" being typed out and the problem itself being repeated. 

A second error on the same problem was followed by the message wrong, 
the answer is the correct answer being displayed. The problem 

itself was then given once more to allow for a correction response. An 
error on the correction response would cause the previous message to 
reappear. The next problem would then be presented. A 10- second time 
limit per response was set. If a response was not given before 10 seconds 
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the machine response followed the above pattern except that the words 
"time is up" were substituted for "wrong" at each step. The sequence 
of events is presented in Figure 1 below. 

Insert Fig, 1 about here 

When the lesson was completed, the machine printed out for the student 
the following: total errors (giving the number of errors), problems 

missed (giving the number of each problem on which an error or time- 
out was made), and total elapsed time in seconds. Following this infor- 
mation it then typed "tear off here _____ __ __ " and turned the paper 

up to the cutter bar permitting the student to tear off and keep the 
printed record of his day's work. 

At the end of the day or when all the children had finished the 
teacher typed the word "finished." The computer program then gave her 
( l) the number of students who made time-outs and errors on each problem 
in the lesson; (2) the distribution of error for the entire class, i.e., 
the number of students making 0, 1, 2, etc., errors and time-outs, and; 

(3) the distribution of the total elapsed time, by 30 second intervals 
for the class. 

Teacher Pre par at i on . Little teacher preparation was required. A 
simple dial-in code was all that was needed to call up each day's lesson. 
The code, consisting of ten steps, was posted on the machine for the 
teacher to follow. 

The Class , The class consisted of 4l fourth-grade pupils. Their 
average IQ on the CTMM was 130. They adapted very quickly to the 
machine. There were few, if any, problems after the second day with 
either the machine operation or finding the right key. 

Supervision . One project staff member was either present or on 
call by phone at all times. Constant attention was required at the 
beginning of the experiment. However, the number of breaking- in problems 





Figure 1. Flow Chart of the Program Logic for Computer-based 
Instruction in the Fundamentals of Arithmetic 
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of the new operation soon diminished and after the first three weeks the 
teacher controlled the daily operation alone, without difficulty. 

Lesson Content , The data presented in this paper are based on 
15 lessons. Prior to these 15 lessons, each student had been given 
2 practice lessons in order to make him familiar with the equipment. 

Lesson 1 contained problems of the form (6 X 7 ) + 3 = , 

(57 - 3) 6 = , and three problems of the form 53 - 4 = 7 X • 

Lesson 2 concentrated on problems of the form (8x6) - 6 = 6 x 

and (7 X 5 ) + 5 = 4 x . Lesson 3 was on units of measure containing 

items such as 1 quart = pints, 1 mile = feet, and 

2 yds. + 5 ft. =3 yds, and ft. Lesson 4 contained such problems 

as (4 x 3 ) x 3 = 9 147 x 4,352 = 4,352 x , and 

3 x (4 + 7 ) = (3x4) + (3 x ). Beginning with Lesson 5, the prob- 

lems were of a simpler form. Lessons 5> 6, and 7 were on the multiplica- 
tion tables for 8, 9, and 10. Each problem had the general form 

a x b = . Lesson 8 was a mixed drill using the operations of addition, 

subtraction, multiplication, and division. Lessons 9 and 10 concentrated 

on multiplication by 10 and 100. Lesson 11 consisted of 5"Word problems. 

This was the first time the children had been exposed to word problems 

on the teletype. One of the problems asked students to find distance, 

given rate and time; two problems required simple mental division; one 

was a simple subtraction problem involving money; and the other was an 

addition problem concerned with tickets to a school play. Lessons 12, 

13 f and 14 were mixed drills containing a large proportion of simple 

multiplication problems. Lesson 15 was another drill on units of 

* 

measurement . 



Findings 

The data obtained from this procedure were summarized on the basis 
of the first answer given by the pupil. If his answer was wrong or 
timed out it was regarded as such, regardless of the response he made 



on his second attempt at the problem. Our main reason for distinguishing 
between errors and time-outs was that if a pupil was timed out, it was 
clear that he had not completed the problem in the allotted time, but 
uncertain whether or not he would have subsequently made an error, had 

he been given more time. 

For each lesson, the average proportions of errors, successes and 
time-outs were computed. The result of this computation is shown in 
Fig. 2, Since the drills varied considerably in the type of problem 



Insert Fig, 2 about here 
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that predominated, this graph is hard to interpret. About all that can 
be said is that there is a tendency for the proportion of successes to 
increase, although there are many obvious exceptions. Owing to this 
somewhat serious confounding between time and lesson-type, a more 
meaningful comparison can be made by rank ordering the lessons in terms 
of proportion of correct responses and classifying them in termsof 
predominant type of problem. The result of an analysis of this type is 
shown in Table 1. In this Table, the predominant form is defined as the 

Insert Table 1 about here 

form of at least three-quarters of the problems in a given lessen. The 
symbols o and o' denote arbitrary but distinct operations. 

It would appear from this Table that the difficulty of a lesson is 
related to the type and predominant form of the problems in the lesson. 
The extent to which the difficulty of a lesson is related to the amount 
of previous practice is impossible to determine. It should be noted 
that, in addition to the problem of differing lesson types, any 
sequential analysis is certain to be confounded by the fact that the 
number of pupils that took each lesson varied. This ' m be seen by 
comparing Lessons 3 and 15, The two are of the same type, but do not 
differ in difficulty (as measured by proportion of correct responses), 
despite the fact that one was presented on the third day and the other 
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Figure 2. Probability of Response Per Pupil on 15 Lessons 



Table 1. Rank order of lessons according to mean 
probability of a correct response. 
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Feasibility of the System 

Teacher Reaction . Although only one classroom teacher -was directly 
involved in this project, her reaction to this project and the machine 
was very encouraging. She remained enthusiastic throughout the seven 
weeks the project was in operation. For example, she made it possible 
to extend the running time through the morning recess or lunch hour on 
days when drills were difficult so that every student could participate. 

She discussed, with individual students, errors on their print outs and 
sometimes modified her instruction to handle problems of general concern 

to the class. 

Limitations Imposed by the System on the Class and Curric ulum. 

Time was an important factor in daily operations during the project. 

The originally planned ^-minute allotment for each student was difficult 
to maintain. Frequently students would run over the planned time, 
causing the day's run to extend through the lunch hour and into the 
arternoon. Length and difficulty of each lesson were the major factors. 
Delay between students increased total running time in some cases. 
Generally, however, there was little delay between students due to good 

management by the classroom teacher. 

The lesson material itself was limited to the characters availaole 
on a standard typewriter. This was not a severely limiting factor, 
however, due to the type of material used in the project. Special print 
wheels with more mathematical symbols will be available for next year, 
greatly increasing the flexibility of the system. The material was 
presented in linear form, requiring every student to do the same lesson 
each day. The possibility of branching to either more or less difficult 
material based on performance was not available. This feature will be 
part of next year's program. 

System Operation. The major source of system failures or operational 
delays was in Stanford's Laboratory itself. Failure to maintain priority 
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Table 2. Individual differences between 
selected students. 
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Feasibility of the System 

Teacher Reaction * Although only one classroom teacher -was directly 
involved in this project, her reaction to this project and the machine 
was very encouraging* She remained enthusiastic throughout the seven 
weeks the project was in operation* For example, she made it possible 
to extend the running time through the morning recess or lunch hour on 
days when drills were difficult so that every student could participate* 
She discussed, with individual students, errors on their print outs and 
sometimes modified her instruction to handle problems of general concern 

to the class* 



Limitations imposed by the System on the Class and Curriculum * 

Time was an important factor in daily operations during the project* 

The originally planned 4-minute allotment for each student was difficult 
to maintain* Frequently students would run over the planned time, 
causing the day* s run to extend through the lunch hour and into the 
arternoon. Length and difficulty of each lesson were the major factors. 
Delay between students increased total running time in some cases* 
Generally, however, there was little delay between students due to good 

management by the classroom teacher* 

The lesson material itself was limited to the characters available 
on a sta ndar d typewriter. This was not a severely limiting factor, 
however, due to the type of material used in the project* Special print 
wheels with more mathematical symbols will be available for next year, 
greatly increasing the flexibility of the system* The material was 
presented in linear form, requiring every student to do the same lesson 
each day* The possibility of branching to either more or less difficult 
material based on performance was not available* This feature will be 
part of next year’s program* 
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System Operation * The major source of system failures or operational 
delays was in Stanford’s Laboratory itself. Failure to maintain priority 
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to access memory in storage accounted for several three- to five-minute 
delays. Computer component failures accounted for 80 percent of the time 
lost. One day was lost due to a telephone- line transmission problem. 
Reliability is expected to improve as the system completes its "breaking- 
in" phase. 

Possibilities for Controlled Classroom Experimentation . A system, 
such as the one used in the project, provides an opportunity for 
controlled classroom experimentation. In many ways it resembles a 
psychological laboratory in providing the possibility of establishing 
strict control over many variables. The data reported in the present 
report represent a very superficial beginning and are intended only to 
give a sense of the methods and procedures that may be used for extensive 
pedagogical and psychological investigation of arithmetic skills. 

Future Curriculum Plans 

The project will be expanded during 1965-66. Three teletypes on a 
full-time basis will be used, one each in grades k, 5 , and 6. Arrange- 
ments are being made at the school to allow easy access, for 80 students 
a day, to each machine. Data analysis and lesson programs are being 
improved to provide more extensive reports to students and teachers as 
well as branching capabilities based on student performance. Each 
student may be given one of five lessons each day depending on his 
performance on the previous day. The five possible lessons vary widely 
in difficulty, the aim being to provide a "floor" for the poor learners 
and a very high "ceiling" for the fast learners. The complete program 
and lesson assignment will be automatic, all necessary information being 
held in the computer bulk memory storage. 
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